Data product generation
mission specific
APXS
The APXS EDR data products are generated by the MIPL (Multimission Image Processing Laboratory) at JPL under the OPGS using the telemetry processing software, mertelemproc. The EDR data products are raw uncalibrated data reconstructed from telemetry data products and formatted according to this EDR SIS. Meta-data acquired from the telemetry data headers are used to populate the PDS label.
There are not multiple versions of an APXS EDR. If telemetry packets are missing during the initial downlink from the rover memory, partial data sets are created and the missing data are filled with zeroes. The data are reprocessed after all data are received and the original version is overwritten.
The APXS memory is maintained by a battery, and in theory, retransmits from the APXS memory could be commanded. In this instance, a new data product is created, because there will likely be subtle and potentially useful changes in the data when a retransmit from the APXS memory is commanded.
Mössbauer
The MB EDR data products are generated by the MIPL (Multimission Image Processing Laboratory) at JPL under the OPGS using the telemetry processing software, mertelemproc. The EDR data products are raw uncalibrated data reconstructed from telemetry data products generated by the SSW team and formatted according to this EDR SIS. Meta-data acquired from the telemetry data headers are used to populate the PDS label.
There are not multiple versions of an MB EDR. If telemetry data is missing, partial data sets are created and the missing data are filled with zeroes. The data are reprocessed after all data are received and the original version is overwritten.
RAT
RAT EDRs are generated by the MIPL (Multimission Image Processing Laboratory) under the OPGS using the telemetry processing software "mertelemproc" at JPL. The EDRs produced are raw uncalibrated data reconstructed from telemetry data products and formatted according to this EDR SIS. Meta-data acquired from the telemetry data headers and a meta-data database is used to propagate the PDS label.
There are not multiple versions of a RAT EDR. If telemetry packets are missing during the initial downlink, the missing data are identified and reported for retransmission of the data to the ground. The data are reprocessed after all data is received and the original version is overwritten.
Mini-TES
The Mini-TES EDRs are generated by JPL's Multimission Image Processing Laboratory (MIPL) under the OPGS using the telemetry processing software "mtes2edr" provided by ASU at JPL. The EDRs produced are raw, uncalibrated data reconstructed from telemetry data products and formatted according to the EDR format defined in this SIS. Meta-data acquired from the telemetry data headers and a meta-data database are used to populate the PDS label.
There are not multiple versions of a Mini-TES EDR. Missing packets are identified and reported for retransmission to the ground as "partial datasets". Prior to retransmission, the missing EDR data are filled with zeros. The EDR data are reprocessed only after all "partial datasets" are retransmitted and received on the ground. In these cases, the original EDR version are overwritten.
Camera Payload Instruments
MER Camera Payload EDRs and RDRs are generated by JPL's Multimission Image Processing Laboratory (MIPL) under the OPGS subsystem of the MER GDS. EDRs and RDRs are also generated by the Athena Pancam Science and Microscopic Imager Science Teams under the SOAS subsystem of the MER GDS.
There are two types of nominal EDR data products generated during the mission, and their applications characterize them as the "Operations EDR" and the "Science EDR".
As the fundamental operations image data archive product, the Operations EDR is generated as "raw" uncalibrated data within an automated pipeline process managed by MIPL under OPGS at JPL as part of the critical path in rover traversability operations. The size of an Operations EDR data product is approximately 2 MB. The total estimated volume of Operations EDRs over the course of the nominal 90-day MER mission is approximately 35 Gigabytes.
The Operations EDR is formatted according to this SIS, such that each of the two radiometric formats of telemetry data are stored "unscaled" in a signed 16-bit integer. In the first case, 12-bit data scaled onboard to 8-bit via a "12 to 8-bit" Lookup Table (LUT) or, by bit shifting, are downlinked as 8-bit data and stored "as is" in the eight lowest bits of the signed 16-bit integer. In the second case, 12-bit data without onboard LUT scaling or bit shifting are downlinked as 12-bit data and stored "as is" in the 12 lowest bits of the signed 16-bit integer.
There are not multiple versions of the Operations EDR. Missing packets are identified and reported for retransmission to the ground as "partial datasets". Prior to retransmission, the missing Operations EDR data are filled with zeros. The Operations EDR data are reprocessed only after all "partial datasets" are retransmitted and received on the ground. In these cases, the original Operations EDR version is overwritten, retaining only a single version.
As the fundamental science image data archive product, the Science EDR is generated by the Athena Pancam Science and Microscopic Imager Science Teams under SOAS at JPL to recover the original 12-bit raw measurement obtained by the respective science camera to within the uncertainty of the noise in the original measured value. The size of a Science EDR data product is approximately 2 MB. The total estimated volume of Science EDRs over the course of the nominal 90-day MER mission is less than that of the Operations EDRs, and depends on the definition of the Science EDR archive set.
There are two cases of the Science EDR. If the Operations EDR is effectively in 8-bit format (e.g., valid data only in the eight lowest bits of the 16-bit integer) because a user-defined "12 to 8-bit" LUT was applied onboard prior to downlink, then a corresponding Science EDR is generated by applying the appropriate Inverse LUT (ILUT) to the Operations EDR. If the Operations EDR is in 12-bit format (e.g., valid data in the 12 lowest bits of the 16-bit integer) because no LUT was applied onboard, it is copied without change in binary data content and renamed as a Science EDR to complete the uniform set of all Science EDRs.
The SOAS software performs some minor reformatting of the PDS label, such as adding spaces for improved readability, breaking up long text strings across multiple lines, etc. The Science EDR shares the same file naming convention as the Operations EDR, altering only the Product Creator character in the filename to "C" (for Athena Pancam Team, Cornell University) or "F" (for Microscopic Imager Team, USGS, Flagstaff) to differentiate it from the "M" (for MIPL) in the Operations EDR filename.
Like the Operations EDR, there are not multiple versions of the Science EDR.
RDR data products are generated by, but not limited to, MIPL using the Mars Suite of VICAR image processing software at JPL, the Athena Pancam Science Team using IDL software at Cornell University and JPL, and the Microscopic Imager Science Team using ISIS software at USGS (Flagstaff) and JPL. The RDRs produced are "processed" data. The input is one or more Camera EDR or RDR data products and the output is formatted according to this SIS. Additional meta-data may be added by the software to the PDS label.
There may be multiple versions of a MER Camera RDRs.
The following is an abbreviated description of the data product generation for the various Phoenix instruments. For more in depth information on data product generation, see the Instrument Software Interface Specifications (SISs).
ASE
The spacecraft contractor, LMCO, was responsible for the supply of the raw telemetry packet data to the Phoenix Science Team. EDRs/RDRs were produced by the Phoenix Science Team cognizant scientist. Raw ASE data packets were extracted from the telemetry stream and stored in data records, by product type and record length.
Camera
As the fundamental image data archive product, the EDR will be generated as “raw” uncalibrated data within an automated pipeline process managed by MIPL under OPGS at JPL as part of the critical path in RA operations. The size of an EDR data product is approximately 2 MB.
RDR data products will be generated by, but not limited to, MIPL using the Mars Suite of VICAR image processing software at JPL, the SSI and RAC science instrument teams using TAMCAL and RACCAL software at the SOC facility at the University of Arizona and at the teams’ home institution at Texas A&M Univerisity, and the MECA Science Team using same set of SSI/RAC software tools at JPL. The RDRs produced will be “processed” data. The input will be one or more Camera EDR or RDR data products and the output will be formatted according to this SIS. Additional meta-data may be added by the software to the PDS label. There may be multiple versions of a PHX Camera RDRs.
MECA - Non-Imaging EDRs
The MGSS-IOS element, supported by the Multi-mission Image Processing Lab (MIPL) at JPL is responsible for generation of the MECA non-imaging EDRs. Raw MECA data packets are extracted from the telemetry stream and stored in EDRs, by activity, product type and record length.
AFM
AFM RDR data products will be generated by the MECA Science Team using software at the Science Operations Center (SOC), the Jet Propulsion Laboratory (JPL) or their home institutions. The RDRs produced will be “processed” data (NASA Level 1). The input will be one or more MECA non-imaging EDR or RDR data products and the output will be formatted according to this SIS and consists of a data file with a .TAB file extension, a PDS label file that has the same name as the data file with a .LBL file extension and a text (.TXT) file that contains information about how the data was collected. Additional meta-data may be added by the software to the PDS label or the data product header table.
The two scan data AFM RDR data products are formatted to have a detached ASCII PDS label. The SDR and SDD data products consist of five attached data tables. The first table is the header table that describes the AFM scan parameters and other important information pertaining to that scan, followed by the calibrated scan data in four sequential ASCII TABLE objects.
TECP
TECP RDR data products will be generated by the MECA Science Team using software at the SOC, JPL or their home institutions. The RDRs produced will be “processed” data (NASA Level 1). The input will be one or more MECA non-imaging EDR or RDR data products and the output will be formatted according to this SIS. Additional meta-data may be added by the software to the PDS label or the data product header table.
There are four types of TECP data, electrical conductivity (designated EC), humidity (designated HUM), relative permittivity or dielectric constant (designated PRM) and temperature (TC). The EC data is formatted as six ASCII tables, a general comments table, an EC comments table, a conversions table, two tables with conversion constants, and a data table that contains a time-series of measurements. The HUM, PRM and TC data are all formatted as a general comments table, a data type specific comments table, a conversions table and a data table that contains a time-series of measurements. The conversions table contains the DN to physical unit equations for the particular data type. In some equations the abbreviation ADC is used for Analog to Digital Converter and is equivalent to DN.
WCL
WCL RDR data products will be generated by the MECA Science Team using software at the SOC, JPL or their home institutions. The RDRs produced will be “processed” data (NASA Level 1). The input will be one or more MECA non-imaging EDR or RDR data products and the output will be formatted according to this SIS. Additional meta-data may be added by the software to the PDS label or the data product header table.
MET
MET data products will be generated by the MET Team led by Co-Investigator Whiteway at York University. The majority of the data processing for the MET-Lidar instrument is performed in the flight segment. The MET-Lidar takes measurements and nominally saves the data to instrument internal Flash memory (Record state). The system will be separately commanded to transmit the data immediately to the Lander (Transmit state). Raw telemetry data are received by the University of Arizona. A, and a number of automated computer processes are run to place data in the MET directory of the UA Science LAN in an ASCII formatted file. The MET-GDS software appends the Timestamp, and writes the data out to files. These are the Level 2 EDRs.
SSI
The Mars atmospheric opacity data products are produced by the SSI instrument team using processing procedures and software developed by Mark Lemmon, Texas A&M Univ.
TEGA
TEGA data products will be generated by the TEGA Team led by Co-Investigator Boynton at the Lunar and Planetary Laboratory, University of Arizona. Once a TEGA measurement is collected, it is stored on the Lander and held for periodic download. The stored telemetry data are downloaded periodically from the Lander for relay to the Deep Space Network (DSN). Data received from the DSN are inserted into the Jet Propulsion Laboratory’s (JPL) Telemetry Data System (TDS). The University of Arizona (UA) queries the TDS for the most recent telemetry dataset. The dataset is output to a spooler that passes data to the UA. Raw telemetry data are received by the UA, and a number of automated computer processes are run to ingest the data into a database, and to transform the data into scientifically useful data products.